Papers with Reading Comprehension
FQuAD: French Question Answering Dataset (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in the field of language modeling have improved state-of-the-art results on many natural language processing tasks. |
| Approach: | They propose to use a French Question Answering Dataset to track progress of French Question answering models. |
| Outcome: | The proposed model achieves an F1 score of 92.2 and an exact match ratio of 82.1 on the test set. |
FilBench: Can LLMs Understand and Generate Filipino? (2025.emnlp-main)
Copied to clipboard
Lester James Validad Miranda, Elyanah Aco, Conner G. Manuel, Jan Christian Blaise Cruz, Joseph Marvin Imperial
| Challenge: | Despite impressive performance of LLMs on English-based tasks, little is known about their capabilities in specific languages such as Filipino. |
| Approach: | They propose a benchmark to evaluate LLMs across a diverse set of tasks and capabilities in Filipino, Tagalog, and Cebuano. |
| Outcome: | The proposed benchmark reflects the priorities and trends of research in the Philippines . it finds that several LLMs suffer from reading comprehension and translation capabilities . |
DACIP-RC: Domain Adaptive Continual Instruction Pre-Training via Reading Comprehension on Business Conversations (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been used in real-world industrial scenarios for various natural language processing tasks, but their high inference cost makes their deployment impractical, necessitating the use of smaller models. |
| Approach: | They propose a continual pre-training technique that generates diverse task instructions and responses via reading comprehension on conversation transcripts, enabling better instruction generalization. |
| Outcome: | The proposed technique improves small LLMs’ domain adaptability for business conversational tasks, compared with traditional methods that rely on next-token prediction. |
FQuAD2.0: French Question Answering and Learning When You Don’t Know (2022.lrec-1)
Copied to clipboard
| Challenge: | Question Answering, including Reading Comprehension, has seen significant scientific breakthroughs over the past few years . but most of these breakthroughs are centered on the English language . |
| Approach: | They propose a dataset to train Question Answering models in the French language . they extend the dataset to 17,000+ unanswerable questions annotated adversarially . |
| Outcome: | The proposed dataset makes it possible to train French Question Answering models with the ability to distinguish unanswerable questions from answerable ones. |
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis (N19-1)
Copied to clipboard
| Challenge: | Existing work on question-answering has limited training examples for RRC . question-announced questions are a key component of online commerce . |
| Approach: | They propose to turn customer reviews into a large source of knowledge that can be exploited to answer user questions. |
| Outcome: | The proposed approach improves review reading comprehension on popular language model BERT . it also improves aspect extraction and aspect sentiment classification tasks . |
STARC: Structured Annotations for Reading Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | STARC is an annotation framework for reading comprehension with multiple choice questions . existing annotation frameworks have limited support for reading behavior analyses . |
| Approach: | They propose a new annotation framework for assessing reading comprehension with multiple choice questions . they use a span ablation dataset to demonstrate that it can be leveraged for a key new application . |
| Outcome: | The proposed framework can be leveraged for a key new application for SAT-like reading comprehension materials. |
MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing reading comprehension metrics rely on token overlap and are agnostic to the nuances of reading comprehension. |
| Approach: | They propose a benchmark for training and evaluating generative reading comprehension metrics: MOdeling Correctness with Human Annotations. |
| Outcome: | The proposed benchmark outperforms baseline metrics by 10 to 36 absolute Pearson points on held-out annotations. |
The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural network models have significantly pushed forward performance on natural language processing benchmarks with the development of largescale language model pre-training. |
| Approach: | They find that models on natural language inference and reading comprehension are unstable . they propose to use a model-selection routine to analyze the model's instability . |
| Outcome: | The proposed models can perform poorly on two language-related tasks, the authors show . they also show that the model selection routine is unstable, and that it is not reliable . |
ScholarlyRead: A New Dataset for Scientific Article Reading Comprehension (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on MRC on scholarly articles have focused on general domain datasets of news articles and elementary school-level storybooks. |
| Approach: | They propose to generate automatic questions from span-of-word-based scholarly articles’ Reading Comprehension dataset with approximately 10K manually checked passage-question-answer instances. |
| Outcome: | The proposed model yields the F1 score of 37.31% and is useful for building Question-Answering (QA) systems on scientific articles. |
Probing and Boosting Large Language Models Capabilities via Attention Heads (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to identifying capabilities rely on external signals with limited structural grounding . emergence of specific capabilities remains poorly understood . |
| Approach: | They propose a lightweight approach that links LLM capabilities to internal components by identifying correspondences at the level of attention heads. |
| Outcome: | The proposed approach improves accuracy on MMLU and BBH by 1 to 1.5 points over gradient-based method and 5 to 6 points over other intermediate-state baselines. |